Efficacy of a Short Role-Play Training on Breaking Bad News in the Emergency Department

Introduction Breaking bad news (BBN) in the emergency department (ED) represents a challenging and stressful situation for physicians. Many medical students and residents feel stressed and uncomfortable with such situations because of insufficient training. Our randomized controlled study aimed to assess the efficacy of a four-hour BBN simulation-based training on perceived self-efficacy, the BBN process, and communication skills. Methods Medical students and residents were randomized into a 160-hour ED clinical rotation without a formal BBN curriculum (control group [CG], n = 31) or a 156-hour ED clinical rotation and a four-hour BBN simulation-based training (training group [TG], n = 37). Both groups were assessed twice: once at the beginning of the rotation (pre-test) and again four weeks later. Assessments included a BBN evaluation via a simulation with two actors playing family members and the completion of a questionnaire on self-efficacy. Two blinded raters assessed the BBN process with the SPIKES (a delivery protocol for delivering bad news) competence form and communication skills with the modified BBN Assessment Schedule. Results Group-by-time effects adjusted by study year revealed a significant improvement in TG as compared with CG on self-efficacy (P < 0.001), the BBN process (P < 0.001), and communication skills (P < 0.001). TG showed a significant gain regarding the BBN process (+33.3%, P < 0.001). After the training, students with limited clinical experience prior to the rotation showed BBN performance skills equal to that of students in the CG who had greater clinical experience. Conclusion A short BBN simulation-based training can be added to standard clinical rotations. It has the potential to significantly improve self-efficacy, the BBN process, and communication skills.


Population Health Research Capsule
What do we already know about this issue? A lack of training in breaking bad news (BBN) leads to inappropriate communication behaviors. BBN protocol has been developed to help physicians to deliver bad news.
What was the research question? What are the effects of a role-play on the BBN process among students who rotated in the emergency department?
What was the major finding of the study? A short BBN simulation-training has the potential to improve the BBN process and communication skills.

METHODS
The ethics committee approved the study (reference number 2015/235). Only authorized individuals had access to the data and materials. The researchers did not participate in the training program.

Control Group
The control group (CG) followed the traditional 160-hour ED rotation. Trainees cared for ED patients under the supervision of EPs. The CG did not receive any formal BBN training.

Training Group
The training group (TG) received a traditional 156-hour ED rotation and four hours of BBNSBT.

BBNSBT
Participants were split into small groups up to six members. The BBNSBT involved two components: 1) a one-hour theoretical course on BBN, SPIKES and communication skills with a 15-minute video illustrating SPIKES components; and 2) a three-hour simulation including six role-plays. Three participants were included in each role-play (one playing the physician and two playing family members) while the three other participants watched the simulation. Each one took 10-15 minutes plus 20-25 minutes for a debriefing. The debriefings followed the framework for Promoting Excellence and Servotte and Bragard et al. Efficacy of a Short Role-Play Training on Breaking Bad News in the ED Reflective Learning in Simulation, using the advocacy-inquiry technique. [36][37][38][39][40] The debriefings focused on the SPIKES protocol and effective communication behaviors.
The following steps ensured the consistency of the BBNSBT: 1) the International Nursing Association for Clinical and Simulation Learning (INACSL) Standards of Best Practice for Simulation SM 41,42 were used to design the BBNSBT; 2) six experts including psychologists, EPs, and simulation instructors validated the scenarios and simulation design; 3) the same facilitators, a psychologist and an EP trained in BBN and certified as basic simulation instructors conducted training; 4) PowerPoint slides with major theory points accompanied the theoretical part of the BBNSBT; and 5) prewritten scripts were used for the roleplay explanations and the debriefings.

Recruitment
Medical students and first-year residents specializing in emergency medicine (EM) who had recently graduated were included in the study for one academic year, between September 2017-June 2018. A convenience sample was invited to participate in the study. It included medical students (n = 64) following a one-month ED internship and first-year EM residents (n = 9) beginning their first month of internship. Each participant gave his or her signed informed consent on a voluntary basis. Five students did not complete the rotation and were excluded from the study; therefore, a total of 68 participants were included. The TG and the CG had, respectively, 37 and 31 members.

Study Design
The feasibility study used cluster randomization to reduce contamination bias. 43 Each month, a group of 10 to 12 medical students and first-year EM residents was randomly assigned either to the TG or the CG. Demographic data such as gender, age, BBN experience, and study year were collected. During the first week of internship, participants underwent a pre-test of their ED BBN self-efficacy and skills. During the second week, participants assigned to the TG participated in the BBNSBT. Post-testing took place four weeks later and included a selfefficacy and ED BBN assessment ( Figure 1).

Assessment Tools Self-efficacy
We assessed the BBN self-efficacy of participants using a seven-item questionnaire (supplemental material 1 specifically developed for this study, corresponding to seven skills (eg, "To manage your nonverbal communication during the BBN"). Participants rated each item on a Likert scale from 1 ("not at all") to 5 ("entirely") in three separate areas: knowledge about the skill; ability to manage the skill; and applying the skill in practice. The experts placed the content validity index of the questionnaire at 0.92. 44 Invitation by e-mail (n=73)  Efficacy of a Short Role-Play Training on Breaking Bad News in the ED Servotte and Bragard et al.

BBN Skills Assessment
BBN skills were assessed in simulation exercises involving two standardized family members played by actors. A randomly selected BBN scenario was used to assess each participant in both pre-and post-test. The scenarios were as follows: 1) a life-threatening situation after a motorcycle accident; 2) a lifethreatening cardiogenic shock; and 3) brain damage after a fight. Each trainee performed in one random scenario. The scenarios for the pre-test and the post-test were different in order to avoid memorization bias. The BBN skills assessments were video recorded and anonymized.
Two blinded raters assessed participants by using two assessment tools. The SPIKES competence form, 28 with 14 items, assessed the participants' compliance with the SPIKES protocol. Each item was scored as "yes" or "no," resulting in an overall score (range 0-14). The experts determined a cut-off score using the modified Angoff method. 45 A passing score was 11 and above, and a failing score was below 11. We used the modified Breaking Bad News Assessment Schedule (mBAS) to evaluate communication. 46 Rather than allocating points proportionally according to the results obtained, the mBAS is reversed, going from 1 (very good) to 5 (very poor). Overall scores ranged from 5-25. A passing score of 14 or lower was also set by the experts. A failing score was above 14.
Assessments were made in two rounds. In the first round, raters independently rated the video. If items were adjacent raw disagreements between raters (more than a one-point difference), they watched the video together, discussed it, and scored it again.

Statistical Analysis
The investigators entered the data collected into the R software, version 3.4.1 (the R Foundation). The statistician used SAS version 9.4 (Cary, NC). We compared the homogeneity of the CG and of the TG at pre-test with χ2 test and Fisher's exact test for qualitative variables and with the Mann-Whitney U test for quantitative parameters.
A generalized linear mixed model 47 (GLMM) measured changes before and after the BBNSBT in self-efficacy, the SPIKES competence form and the mBAS. We adjusted the effects of time, group, and group-by-time by the study year as a confounding factor. GLMMs were performed with a covariance matrix of the compound symmetry type. 48 We performed the McNemar's test to compare the proportion of students who passed the SPIKES competence form and the mBAS cut-offs between pre-test and post-test within the groups.
Furthermore, two further analyses were considered. First, we calculated the relative gains between pre-and post-test within the two groups by means of the following formula: [(post-test -pre-test) / pre-test]. A Mann-Withney U test was used to compare relative gains. Second, we tested whether the BBNSBT could help fill the performance gap between participants with limited clinical experience (less than one year) in the TG and participants with clinical experience (more than one year) in the CG by means of a Mann-Whitney U test. Results were considered statistically significant at the 5% critical level (p< 0.05). Table 1 summarizes the sociodemographic parameters for gender, age, BBN experience, study year, training before BBN simulation training, and pre-test assessment scores. No statistically significant difference was found between the TG and the CG for gender (p = 0.24) and BBN experience (p = 0.44). No participants had attended a communication skills training workshop before the BBNSBT. A statistically significant difference was found for study years (p<0.001). In the CG, participants were predominantly in the third or fourth year of medical school whereas in the TG, they were predominantly in their second year. There was also a statistically significant difference in mean age (p = 0.02), although all students were between 22 and 26 years old.

Pre-test Assessment
Pre-test assessment results are reported in Table 1. We found no statistically significant difference between the TG and the CG with regard to self-efficacy (p = 0.74). However, at baseline the CG had better scores in the SPIKES Competence Form (p = 0.03) and for BBN skills according to the mBAS (p = 0.02). Table 2 presents the results at pre-test and post-test for each group, time effect, group effect, and group-by-time effect. These effects were adjusted by study year. There was a significant group-by-time effect of the training on participants' self-efficacy (p<0.001). Self-efficacy improved significantly over time, with a 55% enhancement for the TG (p<0.001), while it fell slightly in the CG (2.6% reduction; p = 0.5) ( Table  3). The difference between these gains in the two groups was highly significant (p<0.001).

Post-test Assessment
A significant group-by-time effect (p<0.001) of the training on the BBN process was also found for the SPIKES competence form. There was a 33.3% improvement (p<0.001) between pre-test and post-test in the TG while there was no significant gain for the CG. The difference between these gains in the groups was statistically significant (p<0.001). With regard to the measurement of communication skills with the mBAS during BBN, we found a significant group-by-time effect (p<0.001) of the training on communication skills. There was a 23.53% reduction in the non-effective communication skills of the TG participants, but 0% in the CG. The difference between the groups was statistically significant (p<0.001). Table 4 shows the proportion of students deemed competent when cut-off scores were applied to the SPIKES and the mBAS between groups at pre-test and post-test, as At post-test, we found a statistically significant (p = 0.02) difference for the SPIKES cut-off score: the TG had a higher number of participants passing the cut-off score (27 students passed; 73.0%) than the CG (14 passed; 45.2%). More TG students (23 of the 37; 62.2%) passed the mBAS cut-off score than CG students (11 of the 31; 35.5%), but without a statistically significant difference (p = 0.07). While there was a statistically significant improvement in pre-and post-test scores for the TG in SPIKES and the mBAS (p<0.001 for both), we didn't find a significant change in the CG scores (SPIKES: p = 0.92; mBAS: p=0.74).  Efficacy of a Short Role-Play Training on Breaking Bad News in the ED Servotte and Bragard et al.

Clinically Inexperienced Participants Benefiting from Simulation Vs Clinically Experienced with No Simulation Sessions
Further analysis compared participants with limited clinical experience (less than one year) in the TG (n=25) and participants with clinical experience (more than one year) in the CG (n=23) ( Table 5). We found no statistically significant difference between subgroups with respect to perceived self-efficacy in pre-test (p=0.13). In post-test, the difference between the two subgroups was highly significant (p<0.001). The self-efficacy of students with limited clinical experience benefiting from simulation was higher than in the more experienced group. While BBN skills were statistically higher in students with clinical experience in the CG at pre-test, (p=0.049), we observed no differences post-test in the two subgroups (p=0.34). The results showed that participants with limited clinical experience made up the difference. Analyses showed the same for communication skills during BBN.

DISCUSSION
Medical educators aim to identify the best methods to prepare students for clinical practice. Traditional training is the common pedagogical method for learning clinical skills. 49 Trainees rarely learn BBN in real clinical practice due to the paucity of opportunities 32,50 51 define the "simulation zone" as areas in which simulation education may be better suited than other methods. BBN is an example of the HALO quadrant: high impact on the patient and low opportunity to practice. This feasibility study assessed the impact of a four-hour ED BBNSBT compared to clinical internship. It was hypothesized that BBNSBT would have the potential to increase participant self-efficacy in BBN communication and management, adherence to BBN stages and processes, and to improve communication skills during BBN. Our results revealed that this training increased self-efficacy perception. Participants had a low level of self-efficacy in pre-test. After the BBNSBT, the TG reported being more confident about their knowledge and application of BBN and about their ability to perform BBN compared to the CG. This confirms the results of another, smaller study (n = 20), which showed an improvement in confidence and self-efficacy. 52 These findings may be explained by Bandura's social cognitive theory, 53 which suggests four ways to enhance selfefficacy that we identify in the BBNSBT: 1) enactive attainment (performing the action during the role-play simulation); 2) vicarious experience (observing the video and watching other participants in the role-plays); 3) verbal persuasion (facilitators support the students during the debriefings); and 4) psychological safety during the simulations. Moreover, the perceived selfefficacy of students in the CG with more clinical experience decreased. This result could have different potential explanations, notably that the pre-test may have led to introspection and reflection about their BBN and communication skills.
Communication with patients and their families is one of the Accreditation Council for Graduate Medical Education Milestones for EM residents, specifically the fourth level of BBN. 54,55 Our research used two validated assessment tools that allow for standardization of the evaluation and training. The results demonstrate that BBNSBT using role-playing and debriefing enhances participant BBN learning and performance compared with the traditional learning paradigm and direct immersion in acute clinical situations. BBNSBT offers the opportunity to teach BBN and communication skills to students and young residents in a psychologically safe environment, preventing harm to patients and family members. It allows each participant to announce bad news and observe several BBN simulations with debriefings.
By contrast, in the traditional curriculum role modeling at the bedside could have a negative impact on patients and relatives when medical students or residents engage in inappropriate communication behaviors, 56,57 such as not keeping patients or family members adequately informed or using medical words they do not understand. More students in the TG reached the cut-off scores: 73% for SPIKES and 62.2% for the mBAS vs 45.2% and 35.5% in the CG. These results demonstrate the relevance of BBNSBT in communicating bad news in the ED. However, the difference between the groups for the mBAS cut-off score is not significant. BBNSBT probably focuses more on SPIKES than on communication behavior. It may be necessary to create an advanced course centered on communication skills rather than on SPIKES.
Despite this, BBNSBT offers experiential learning for participants. From the simulation experience, the debriefing process leads students to explore their frames, incorporate new frames such as SPIKES skills, and re-practice these new skills. This process allows knowledge to be acquired through experience. 56 Moreover, participants had access to ED BBN experts for four hours, which, unfortunately, is unlikely to happen in real clinical practice.
Additional data analyses allowed us to address a new question: Is BBNSBT more useful for students with less  than one year of clinical experience? We found a statistically significant difference in the pre-test. Students with limited clinical experience reached the same level of BBN skills as students with more clinical experience after the BBNSBT. The gap between these groups could be filled by simulation training, without the pitfalls of stress and discomfort of direct clinical exposure. No study has previously focused on this question. In fact, BBNSBT used a step-by-step process involving novice participants to bring them to a higher level. The first step involved theoretical explanations given via video, discussions, and lectures. Each simulation, and especially each debriefing, further enhanced the participants' skills.
One strength of the study is that we paid special attention to the theoretical background upon which the training and evaluation were based, using the widespread SPIKES 28 theoretical model and the INACSL Standards of Best Practice for Simulation SM . 41,42 Moreover, the simulations were well designed, the debriefings were standardized, and the facilitators were trained and experienced. We believe that it is mandatory to meet the INACSL Standards of Best Practice, as well as work with simulation experts to obtain positive results with simulation training.
The next steps for research and pedagogical method improvement can be identified based on these results. Further research is needed to investigate the role of an advanced course in BBN. As BBN is not a required skill for EPs, it would be interesting to investigate whether BBNSBT is feasible and effective in other areas such as obstetrics, intensive care units, etc. Finally, we think that e-learning preparation before BBNSBT, as described for a training on managing low urine output, 58 could replace some of the in-person time.

LIMITATIONS
According to Kirkpatrick's four-level training evaluation model, 59 the self-efficacy and skills assessment used in the simulation are categorized at level 2, which is a low level. 60 Moreover, we assessed the impact of BBNSBT just after training. Skills transfer to a real clinical setting is not guaranteed and does not allow for any definite conclusion with regard to the actual impact on patients or family members. 50 Future studies could assess the impact of this training in the workplace and on skills retention over time. 60 Despite these limitations, the results are very encouraging given that training is only four hours long, a significantly shorter period than other programs previously described. 25 While the three scenarios used for the assessments share similarities, they were different before and after the training, as in real life. Cluster randomization resulted in an inequitable distribution of participants. Despite this heterogeneity, statistical analyses adjusted by study year seem to prove that BBN training has an impact on students. This study assessed the impact of a four-hour BBN training, but we cannot be sure that this duration would be more effective than two or six hours. Finally, we did not assess the emotional impact of BBNSBT and BBN assessment on trainees. It would be interesting to know whether BBNSBT elicits a different response than traditional internships.

CONCLUSION
Training programs aspire to produce competent emergency physicians including excellence in the domains of professionalism and communication. According to the EM Milestones, the target for a trainee ready to graduate for "patientcentered communication (ICS1)" specifically includes being able to deliver bad news. The results of this study revealed that a short, simulation-based training with a debriefing session may improve the self-efficacy, BBN skills, and communication skills of medical students and young residents in the ED. Role-playing appears to be an effective and feasible way for trainees to master BBN and acquire patient-centered skills. Further studies should assess the transfer and retention of these skills as well as when to implement the simulation training in the curriculum.